UThe Vision Research Lab of UCSB at TRECVID 2007

The Vision Research Lab at the University of California at Santa Barbara participated in three TRECVID 2007 tasks: rushes summarization, high level feature extraction, and search. This paper describes contributions in the high level feature and search tasks. The high level feature submissions relied on visual features for three runs, audio features exclusively for one, and a fusion of audio and visual for the remaining two; Table 1 provides a summary. Four MPEG7 features (DCD, CLD, EHD, and HTD) comprised the global visual features, and a SIFT signature from a vocabulary tree generates the local-feature representation. It was discovered that the local features performed quite well independently. We combined audio and visual methods as a weighted fusion using SVM scores from the visual features, kNN-derived scores for the visual features, and audio feature SVM scores. Linear fusion using a grid search for weights on the visual features, without audio, is found to perform best. Additionally, we submitted a fused run based on weighted Borda counting on the ranked lists from audio, global visual features, local visual features, and a face feature. This run had similar performance to the weighted fusion that also included audio. All of our runs were type A, only using commonly annotated data for training. Table 1: High Level Feature Submission Summary HLF Run ID MAP Description A UCSB 1 0.051 SVM on local SIFT signature concatenated with global features A UCSB 2 0.049 Borda count fusing local feature SVM scores, global feature SVM scores, audio feature scores, and face detection scores A UCSB 3 0.043 SVM on local SIFT signature A UCSB 4 0.015 Audio-only run A UCSB 5 0.060 Fusion using combination of visual-only kNN and SVM classifiers A UCSB 6 0.050 Fusion using methods from run 5 and audio classifier For the search task we submitted fully automatic baseline text, visual, and concept selection runs, as well as a manual baseline audio-only run and two fusion runs. The visual submission combined lowlevel feature querying with 36-dimensional concept-vector querying. The text run scores were based on text matching between the query and NIST machine-translated transcript, but this submission was not scored. A concept selection technique was developed to select multiple concept detectors from the previous task and from an expanded 374-LSCOM annotation provided by Columbia by expanding the visual and textual queries. One fusion run used Borda count to combine the lists produced individually without any training data; the other fusion technique used a Markov chain based method for list fusion.